Skip to content

[SPARK-24810][SQL] Fix paths to test files in AvroSuite #21773

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
MaxGekk wants to merge 5 commits into from
Closed

[SPARK-24810][SQL] Fix paths to test files in AvroSuite #21773

MaxGekk wants to merge 5 commits into from

Conversation

@MaxGekk
Copy link
Member

@MaxGekk MaxGekk commented Jul 15, 2018

What changes were proposed in this pull request?

In the PR, I propose to move testFile() to the common trait SQLTestUtilsBase and wrap test files in AvroSuite by the method testFile() which returns full paths to test files in the resource folder.

@SparkQA
Copy link

SparkQA commented Jul 15, 2018

Test build #93027 has finished for PR 21773 at commit d6ac418.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

HyukjinKwon
HyukjinKwon approved these changes Jul 15, 2018
View reviewed changes
Copy link
Member

@HyukjinKwon HyukjinKwon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@SparkQA
Copy link

SparkQA commented Jul 15, 2018

Test build #93028 has finished for PR 21773 at commit f9540d2.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

gengliangwang
gengliangwang reviewed Jul 15, 2018
View reviewed changes
external/avro/src/test/scala/org/apache/spark/sql/avro/AvroSuite.scala Outdated
val episodesFile = "src/test/resources/episodes.avro"
val testFile = "src/test/resources/test.avro"
val episodesAvro = "episodes.avro"
val testAvro = "test.avro"
Copy link
Member

@gengliangwang gengliangwang Jul 15, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit:
Maybe we can make it simpler

val episodesAvro = testFile("episodes.avro")
val testAvro = testFile("test.avro")

@SparkQA
Copy link

SparkQA commented Jul 15, 2018

Test build #4212 has finished for PR 21773 at commit f9540d2.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jul 15, 2018

Test build #93033 has finished for PR 21773 at commit 09f0372.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jul 15, 2018

Test build #93035 has finished for PR 21773 at commit 98354ec.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
  • class AvroDeserializer(rootAvroType: Schema, rootCatalystType: DataType)
  • sealed trait CatalystDataUpdater
  • final class RowUpdater(row: InternalRow) extends CatalystDataUpdater
  • final class ArrayDataUpdater(array: ArrayData) extends CatalystDataUpdater
  • class AvroSerializer(rootCatalystType: DataType, rootAvroType: Schema, nullable: Boolean)
  • class IncompatibleSchemaException(msg: String, ex: Throwable = null) extends Exception(msg, ex)
  • class SerializableSchema(@transient var value: Schema)

@HyukjinKwon
Copy link
Member

HyukjinKwon commented Jul 16, 2018

retest this please

@SparkQA
Copy link

SparkQA commented Jul 16, 2018

Test build #93039 has finished for PR 21773 at commit 98354ec.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
  • class AvroDeserializer(rootAvroType: Schema, rootCatalystType: DataType)
  • sealed trait CatalystDataUpdater
  • final class RowUpdater(row: InternalRow) extends CatalystDataUpdater
  • final class ArrayDataUpdater(array: ArrayData) extends CatalystDataUpdater
  • class AvroSerializer(rootCatalystType: DataType, rootAvroType: Schema, nullable: Boolean)
  • class IncompatibleSchemaException(msg: String, ex: Throwable = null) extends Exception(msg, ex)
  • class SerializableSchema(@transient var value: Schema)

gatorsmile
gatorsmile reviewed Jul 16, 2018
View reviewed changes
Copy link
Member

@gatorsmile gatorsmile left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Thanks! Merged to master

zzcclp pushed a commit to zzcclp/spark that referenced this pull request Jul 16, 2018
@MaxGekk @gatorsmile
[SPARK-24810][SQL] Fix paths to test files in AvroSuite
9f92945 
## What changes were proposed in this pull request?

In the PR, I propose to move `testFile()` to the common trait `SQLTestUtilsBase` and wrap test files in `AvroSuite` by the method `testFile()` which returns full paths to test files in the resource folder.

Author: Maxim Gekk <[email protected]>

Closes apache#21773 from MaxGekk/test-file.
@MaxGekk MaxGekk closed this Jul 16, 2018
@MaxGekk MaxGekk deleted the test-file branch August 17, 2019 13:35
otterc pushed a commit to linkedin/spark that referenced this pull request Mar 22, 2023
@MaxGekk
[SPARK-24810][SQL] Fix paths to test files in AvroSuite
a406892 
In the PR, I propose to move `testFile()` to the common trait `SQLTestUtilsBase` and wrap test files in `AvroSuite` by the method `testFile()` which returns full paths to test files in the resource folder.

Author: Maxim Gekk <[email protected]>

Closes apache#21773 from MaxGekk/test-file.

(cherry picked from commit 9f92945)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gengliangwang gengliangwang gengliangwang left review comments

@gatorsmile gatorsmile gatorsmile left review comments

@HyukjinKwon HyukjinKwon HyukjinKwon approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@MaxGekk @SparkQA @HyukjinKwon @gengliangwang @gatorsmile